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ABSTRACT 



Digital video transmission rate matching techniques are 
disclosed. Abit slreamrate matehing £^aratus includes a bit 
rate increasing device and a bit rate reducing device. The bit 
rate increasing device converts a video bit stream having a 
first transmission rate to a video bit stream having a second 
transmission rate wherein the first transmission rate is less 
than the second transmission rate. The hit rate reducing 
device converts a video bit stream having the second trans- 
mission rate to a video bit stream having the first transmis- 
sion rate. The hit stream rate matching apparatus is useful in 
the context of a multimedia conference where a first end- 
point device employs the first transmission rate and a second 
end|point device en^loys the second transmission rate. 

11 Claims, 6 Drawfaig Sheets 
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VTOEO TRANSMISSION RATE MATCHING Existing MCUs require aU endpoint devices participating 

FOR MULTIMEDIA COMMUNICATION in a given nmltimedia conference to use the same video 

SYSTEMS transmission rate. Typically, dunng the initial setting up of 

a multimedia conference, or at the time that it is desired to 
CROSS-REFERENCE TO RELATED 5 add an additional endpoint device to an existing conference, 

APPLICATION the MCU polls all of the endpoint devices to ascertain the 

Related subject matter is disclosed in the co-pending ""'"^^ transmission rate or rates each endpoint device is 
commonly assigned U.S. patent application of: Yan et aL ^"^"^J^ implement When tfie MCU poUs a given 
entitled "Coded Domain Picture Composition for Multime^ cndpomt device, en^int device transmits a capability 
diaCommunications Systems" Ser. No. 08/332,985 andfiled ^° «>'l^ bade to the MCU. The capability code includes infor- 
simultaneously herewith. maUon specifying the video bit rate transmission 

capabilities, audio bit rate transmission capabilities, etc., of 
BACKGROUND OF THE INVENTION * endpoint device. These capability codes may also 

specify the compression algorithm or algorithms used by a 

1. Field of the Invention 15 given endpoint device. 

The invention relates generally to multimedia communi- Based upon the capability codes received by the MCU 
cation systems whidi utilize endpoint devices, and more from the various endpoint devices, tiie MCU selects a 
specifically to techniques for matching the video transmis- minimum video bit rate to be used as the video transfer rate 
sion rates of a plurality of endpoint devices. for the multimedia conference. The MCU sends a video rate 

2. Description of the Prior Art ^ signal to all endpoint devices which specifics use of this 
Video transmission rate matching is a technique which transfer rate. Upon receipt of the video rate signal, Ae 

has been used to convert the bitrate of a first video bit stream ^ii^Joint devices all opiate using the same video transfer 
to match that of a second video bit stream- This oonvcision 

may require bit rate reduction andyor bit rate increases, 25 video transfer rate for all endpoint 

Among these two conversion processes, bit rate reduction is devices participating in a given conference presents signifi- 
more critical, due to the fact tiiat this process involves the shortcomings. For example, in many cases, a confcr- 

removal of bits. Since these bits represent video information, participant may desire to receive high-quality video by 

the quality of a video signal having a reduced bit rate may ^sing a relatively high video transfer rate, whereas other 
be degraded. The amount of degradation is related to the 30 participants may want to use less expensive equ^ment 
manner in which the bit rate reduction is achieved. offering a lower video transfer rate. State-of-the^ MCUs 

With respect to hit rate increases, no infonnation is cannot in^enient a multimedia conference using a pluraUty 
removed from the video bit stream. Instead, such a rate different video transfer rates. Therefore, it would be 

increase requires the incorporation of additional bits into the desirable to equip existing MCUs with a video transmission 
bit streanL Although it would be desirable to add bits to the 35 matching device. Through the use of video transmission 
bit stream for the purpose of enhancing the video signal, matching tediniques, video communication among a 

such enhancements are often in the pel domain. For Plurality of eni^wint devices will be provided, even when 
example, if an incoming video signal has been subjected to endpoint devices use different video transmission rates, 

a process of data compression, the information discarded in video transmission rale matching method well- 

thc compressioD process is not stored. The information is 40 ^own to those skilled in the art is termed the transcoding 
lost, and cannot be recovered and added to the bit stream method. Rirsuant to the transcoding method, a con:q)ressed 
during the process of increasing the bit rate. TherefcH-e, bit video bit stream having a first bit rate is fully decoded into 
rate inacascs are implemented by adding "extra" bits to the ^ video space known as the pel domaiiL This fuUy-decoded 
bit stream, wherein these "extra" hits are not die same hits stream, which may be concqptualizcd as a compdetcly 

that may have been removed during a previous data com- 45 reconstructed video sequence, is then encoded into a video 
pression step. Bits are added using conventional bit-stuffing stream having a second bit rate, 

patterns to occupy the additional bandwidth of a rate- The existing transcoding method is disadvantageous. 
Increased video signal Since decoding as well as encoding processes are required] 

Video transmission rate matching has been applied to a transcoding is vciy time-consumiag. As a practical matter, 
wide variety of applications in the field of multimedia 50 the time delay is at least twice that of the end-to-end 
conununication, especially in environments involving mul- encoding delay. Such a delay is not tolerable for applications 
tipoint communication topologies. For cxan^le, in muM- requiring real-time communication, such as multimedia con- 
media conferencing applications, a **bridgc" or "multipoint ferendng. A faster method of performing video bit rate 
control unif * (MCU) is typically en^loycd. This MCU is a matching is needed. 
computCT-controUed device which includes a multiplicity of 53 

communication porta. The ports may be selectively inter- SUMMARY OF THE INVENTION 

connected in a variety of ways to provide communication Digital video transmission rate matching techniques are 
among a group of endpoint devices. MCUs are often used to disclosed. Abit stream rate matching OT)aranis includes abit 
cstobbsh multi-party conferences among a plurality of end- rate increasing device, a hit rate reducing device, and first 
pomt devices, and to cstabHsh multi-party conference calls. 60 and second switching devices. The first switching device 
t^f^^ described in ITU Document switcheseachof a plurality of incoming video bit sfreams to 

H.243, T^ocedures for EstabUshing Communication any one oftiie bit rate increasing device, the bitrate redudne 
Between Three or More Audiovisual Terminals Using Digi- device, and the second swUching device Each of these 
tal Channels up to 2 Mbps", March 1993, and in ITU incoming video bit streams aiginates from a particular 
Document H.231, "Multipoint Conttol Units f<H: Audiovi- 65 endpoint device. The second switching device switches 
si^ systems Usmg Digital Channels up to 2 Mbps", Mardi video bit streams from the bit rate reducing device, the bit 

rate increasing device, and the first switching device to each 
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of a plurality of outgoing video bit streams. The first and 
second switching devices are controlled by a processor. The 
bit streamrate matching apparatus is useful in the context of 
a video communication where a first endpoint device 
employs the first transmission rate and a second endpoint 
device employs the second transmission rate. 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a hardware block diagram of a video transmis- 
sion rate reduction apparatus according to a preferred 
embodiment disdosed herein; 

FIGS. 2 and 3 are data structure diagrams setting forth 
illustrative coding formats for representing video infonna- 
tion; 

FIG. 4 is a data structure diagram illustrating the arrange- 
ment of DCr coefGcients within one of the blocks shown in 
FEG. 3; 

FIG. 5 is a data structure diagram setting forth an illus- 
trative example of DCT coefficient partitioning; 

FIG- 6 is a software flowchart setting forth a procedure for 
performing video bit rate matching according to a preferred 
embodiment disclosed herein; 

FIG. 7 is a hardware block diagram setting forth an 
illustrative structure for the discrete cosine transformation 
(DCT) processor of FIG. 1; 

FIG. 8 is a hardware block diagram setting forth the 
system architecture of a multimedia conferencing system 
which includes an MCU having a video processor eqitq)ped 
to perform video transmission rate matching methods 
according to various preferred embodiments disclosed 
herein; 

FIG. 9 is a block diagram showing an illustrative hard- 
ware configuration for the video processor of FIG. 8; and 

FIG. 10 is a hardware block diagram setting forth a bit rate 
matching system for use with five endpoint devices. 

DETAILED DESCRIPTION 

For illustrative purposes, video transmission rate match- 
ing will be described in the operational context of an H.261 
environment. However, it is to be understood that the video 
transmission rate matching techniques described herein are 
generally applicable to any video compression algorithm 
whidi uses transfcamation and quantization processes. For 
example, the techniques disclosed herein may be en^loyed 
in conjunction with video telephones of the type described 
in CCUT recommendation COST 211. However, for pur- 
poses of the present disclosure, video data to and/or fromtbc 
video transmission rate matching system will be compressed 
in an H-261-likc format 

An "H^ei-like" foonat is any coding format which is 
similar to the coding format currently being established by 
the International Telecommunications Union (XTU-T). The 
format is described in the TTU-T document "Recommenda- 
tion H.261, Video Codec fo" Audiovisual Services at px64 
kbits/s", May 1992, and the ITU-T document 'Ttescription 
of Reference Model 8", Jun. 9, 1989. 

FIG. 1 is a hardware block diagram showing a transmis- 
sion rate reduction system 100 for a video signal which has 
been compressed in an H.261-like format. The transmission 
rate reduction system 100 includes an input port 101 for 
accepting a compressed video signal having a first bit 
transmission rate and an output port 116 for providing a 
compressed video signal having a second bit transmission 
rate. A compressed video signal is defined as the binary 
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represMitation of a video signal which has been compressed 
by a coding algorithm substantially similar to that described 
in the H.261 standard, and then coded according to a syntax 
substantially similar to that described in the H.261 standard 

5 A compressed video signal in the form of a digital, coded 
bit stream is presented to input port 101. This compressed 
video signal includes coded digitized video infomiatlon 
along with a header. The header may include the types of 
information specified in the H.261 standard, and^or other 
types of information sudi as the source, the destination, the 
content, and/or the organization of the video information. 

Input port 101 is connected to a receiving buffer 102, 
which is a conventional digital buffer. This receiving buffer 
102 provides electrical isolation between the source of the 
conq>ressed video signal and the various circuit dements 
shown in FIG. 1. 

The output of the receiving buffer 102 is coupled to a 
video multiplex decoder (VMD) 104. VMD 104 indudes the 
combination of a decoder and a demultiplexer. The demul- 

2Q tiplexer is equipped to demultiplex the coded bit strcanL The 
decoder decodes header infcimation which has been coded 
into the aforeancntioned coded bit stream. The demult^lexcr 
and decoder perform the function of recovering con^iressed 
video data from the coded bit stream. 

25 The VMD 104 indudes a first output port 105 and a 
second output port 106. The VMD 104 provides the first 
output port 105 with quantized DCT coefficients along with 
quantization information. The nature of these DCT cx}effi- 
cients is described in greater detail in the H.261 standard 

30 The second output port 106 is provided with motion vectors. 
First output port 105 is coupled to DCT coefficients proces- 
sor 107, and second ouqmt port 106 is coupled to a video 
multiplex encoder (VME) 109. The operation and structure 
of VME 109 will be described in greater detail below. The 

35 motion vectors are sent directly from VMD 104 to VME 109 
because no motion estimation is cn^loyed in the present 
example. 

The DCT coeffidents recdved fi*om the first ou^ut 105 of 
VMD 104 are sent to DCT coeffidents processor 107, The 
40 DCT coeffidents processor 107 processes the DCT codOB- 
dents in sudi a manner ttk&t the output 108 of DCT proces- 
sor 107 is a signal which requires fewer bits to encode than 
the numb^ of bits whidi were received from the first output 

105 of the VMD 104. The DCT coeffidents processor 107 
45 reduces the number of bits sudi that the resulting video 

quality is not substantially degraded as perceived by hiunan 
visual processes. In other words, the video quality is 
degraded "gracefully". The amount of information reduced 
by processing coeffidents at DCT coeffidents processor 107 

50 is controlled by the rate-control signal 114 sent by rate 
control unit 113. Various techniques for programming DCT 
coefficients processor 107 to degrade video quality g;race- 
fully are well known to those sldlled in the art. Three such 
techniques for gracefully degrading video quality will be 

55 discussed in greater detail herdnafter. 

The processed DCT coeffidents are produced at the 
ou^ut 108 of DCT coeffidents processor 107. These pro- 
cessed DCT coeffidents, together with the motion vectors 

106 produced at the second output of VMD 104, are sent to 
60 the video multiplexing encoder (VME) 109 to form a new 

video bit stream substantially conforming to the H.261 
standard. The VME 109 encodes the processed DCT coef- 
fidents and motion vectors, and multiplexes them into the 
layered data structures shown in FIG. 3, to be described in 
65 greater detail herdnafter. The new coded bit stream, pro- 
duced at the output 110 of VME 109, is sent to a transmis- 
sion bufier 111. 
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As It IS wdl-known that compressed video data signals simply indices of the DCT coefficients within a macro block, 

may mdude a plurality of components, wherein each com- whose coefficients will subsequently be set to zeroes By 

poncnt may be n^rcscntcd by a different nimiber of bits, forcing some DCT coefficients to zero, the amount of data 

transimssion buffer lUperfonns a variable^ to constant- produced by the DCT coefficients processes lOT can be 
rate franslatiOD for the compressed video. TTie transmission 5 p^opedy controUcd by rate control 113 circuit, 

buffer 111 mcludes circuitry to ascertain and to mdicate the tTn-e ^ ^ ^ ^ 

status of the transmission buffer 111, which is defined as ±t .,1 ■ T f^"^^ <hagrmn3 setting forth 

occupancy ratio of the mcmoiy locations within the tians- J^ustrative coding formats for representing video informa- 

mission buffer HI. The occupancy ratio refers to the ratio accordance with the H.261 standard. Refeiring now 
betweenthenumberofoccupicdmcmorylocationswithina 

given buffer and the total number of memory locations 2^1' ^03, 205, 207, 209, 2 

within this buffer. TTie buffer status is produced at a first f. ^'^^^ a a reprc^ntation of a two- 

input^output U2 of the transmission buffer 111. This first ^^^^^ ^ fonn of a pixel array. Since 

ou^ut 112 is coupled to a rate control 113 circuit This rate ^ represent a video image at a specific 

control 113 circuit adjusts the average data mte provided by ^ PluraUty of frames may be employed to 

the DCT coefficients processor 107? represent a moving image. Each frame represents a picture. 

Rate control 113 drcait and DCT coefficients processor ^P^^^ ^P^^^^^ ^ video sequence. 
107 are coupled together via signal line 114. The rate control ^ frames is cornjprcssed according to any one of 
U3 circuit inchides a first communications line 115 ad^tcd concession algorithms, termed intca-ftame 
for receiving a signal, (for example, from a control 20 ® predictive coding (P). For exan^le, firamcs 
processor) which specifies a desired output bit rate for ^ comi^essed using intra-firame coding (1), 
transmission buffer UL The communications line 115 is ^09, 213, 215, 217, 219, and 221 
also ad^ted to transmit signals to control processor 840 con^ressed using predictive coding. The sequence of 
(FIG. 9). The desired output rate signal is processed by rate ^ ™- ^ establish a data structure for 
control circuit 113 in conjunction witii the buffer status 25 '*T^^^g^.^*^ ^"^^ *^fo™ of an encoded video 
signal received at the first input/output 112, to generate a rate sequence having a plurality of levels arranged in a two- 
control output signal which is downloaded via signal line <*inipnsional array, wherein each level represents the value of 
114 to DCT coefficients processor 107. Based upon the ^ ^^^^ dement This encoded video sequence may be 
desired output rate signal and the buffer status signal, tiie * ^ 

rate control 113 circuitry computes the total number of bits 30 ^ intra-frame coding is to be applied to a given frame, 

for each frame, as well as the bits targeted for each macro fi:ame 201, the firame is termed an I-designated 

block. The targeted bits per macro block or bits per frame arc frame, and if predictive coding (P) is to be ^plied to a given 

used as reference to produce a proper control signal, whidi frame, such as frame 205, the frame is termed a P-designated 

is applied to a second input 114 of the DCT coefficients franic- 

processor 107. The function of the second input 114 is to 35 Pursuant to intra-frame coding (I) compression processes, 

frace the DCT coefficients processor 107 to operate in such the I-designated frame 201 is divided into a plurality of pixel 

a manner that the bits produced for each macro block are as blocks, wheiein each block consists of an array of 8x8 

close to the targeted bits per macro block as possible. The pixels. Next, a discrete cosine transform (hereinafter, DCT), 

type of signal applied to second input 114 is dependent upon is perfdnncd on each of the pfatels in the 8x8 pixel block, in 
the specific type of DCT co^dents processor 107 which is 40 accordance with pnxiedures well-known to those skilled in 

utilized. the art, to generate a plurality of DCT coefficients. 

DCT coefficients processor 107 is the processing unit Thereafter, quantization is performed on the DCT 

where the DCT coefficients recovered from the VMD 104 coefficients, in accordance with well-known quantization 

are further processed in order to match a desired (i.e., target) procedures. These quantized DCT coefficients constitute 

ou^ut video rate. Three methods can be used to process the 45 compressed video image information for the I-cncoded 

DCT coefficients to reduce the total number of bits. Each of frdmc 201. 

thesemethods provides forthc graceful degradation of video Predictive coding (P) is implemented on a P-desicnated 

qj^ty, as was discussed above. The first method is tamed frame, such as frame 205, by: 1) partitioning the 

DCT coefficients zeroing, the second method is called the P-designated frame into a phiraHty of macro blocks For 

requantization of the DCT coefficients, and the third method so example, if the frame includes a plurality of pbtd arrays, 

consists of the combination of the first and second methods. each having 16x16 pixels (FIG. 2, 251 2S2 253 254 257 

In the first metiiod, DCT coefficients are partitioned into 258), each 16x16 pixel array may be partitioned iLto 4 

groups based upon the relative importance of the various contiguous blocks, wherein each blockis an 8x8 pixel anay 

coefficents. Due to tiie fact that DCT coefficients are gen- a 16x16 pixel array Guminance) together with an 8x8 pixel 

crally organizwi into two-dimensional arrays wherein tfie 55 block (chrominance) and an 8x8 pixel block (chrominance), 

array enhics which are rdativdy dose to the upper left-hand comprises a macro block 247; 2) for each of the macro 

corner of the array indudc relatively low-frequency blocks (16x16 pixels) created in step (1), searching Uie most 

components, as con^ared with array entries which are recent previously occurring frame (which could be either a 

relauvely close to the lower right-hand comer of the array, P- or an I-designated frame, but in the present example is 

fte relative importance of various DCT coefficients is 60 frame 203) for the macro block which contains hnage 

known. The lower frequency components are more hr^or- information that is most similar to the image information in 

tant and the higher frequency components are less impor- tiie macro block created in step (1); 3) generating motion 

tant Based upon the output produced by rale control 113 vectors to spatiaUy translate the macro block found in the 

arcmt on signal line 114, the coefficients of the least prior I or P frame in step (2) to the location of die similar 

important group are set to zeroes. Here, the control signal on 63 macro block in the P frame presently being compressed' 4) 

si^al Ime 114 consists of a digital r^esentation of the generating a predicted frame from the most recent prJvi- 

indices of a pluraUty of specific importance groups, or ously occurring frame using die motion vectors- 5) on a 
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macro-block-by-macro-block basis, subtracting the pre- block 247 consists of 6 blocks, including four luminance 

dieted frame from the P-frame being conqiressed, to gener- blocks Yl (reference numeral 251), Y2 (reference numeral 

ate blocks of residues; 6) perfoiming DCT's on the blocks 252), Y3 (reference numeral 253), Y4 (reference numeral 

of residues; 7) quantizing ttie coefficients of the blocks of 254), one chrominance block U (reference numeral 257), 

transformed residues; and 8) concatenating the quantized 5 one dirominance block V (reference numeral 259). An 

residue coefficients and the motion vectors to form a com- illustrative example of the contents of luminance block U 

pressed video sicnaL (reference numeral 257) is set forth in FIG. 2. Note that this 

. ^ , . ._. t . block includes an 8x8 pixel array wherein all pixels have a 

In an mtra-framc coded (1) picture, every macro block is lummance value of black, 

intra-coded. That is, each naaoro block is coded without ^ represents a matrix (array) of pixels, e.g., 8x8, 

rcfemngtoany macro blockmthepreYiousI-crP-franie,I^ lo over which a discrete cosine transfonn (DCT) is performed 

the predictive-coded (P) picture, the macro block can be p^^eis is represented by a matrix of pbiel array 

either intra-coded or inter-coded. coefficients, including a DC coefficient and a plurality of AC 

To form the coded video bit stream for transmission, the coefficients. The transformed coefficients (TCOEFF) 301 

cony)rcssed image information, as well as other information (FIG. 3) consists of a DCT DC coefficient occurring first, 

such as motion vectors, are coded using specified code followed by respective pixel array coefficients (AC), in the 

words. The code words are then multiplexed into a layered order of their relative inaportance. The arrangement of DC 

data structure to form the final bit stream. In an H.26I-like and AC coefficients in an illustrative block data field 277 

environment, the coded bit stream is organized into a (FIG. 3) is shown in FIG. 4. The block data field 277 (FIG. 

hierarchical format, the structure of whidi is illustrated in 3) consists of the transformed coefficients (TCOEFF) 301 

FIG. 3. and an end of block code (EOB) 303 which is upended at 

Referring to FIG. 2, the sequence of frames 201, 203, 205, the end of each successively occurring block of data. 
207, 209, 211, 213, 215, 217, 219. 221 forms a coded video A typical partitioning of DCT coefficients is fllustrated in 
bit stream. This bit stream may be concepmalized as a serial FIG. 5. The DCT coefficients are arranged in a two- 
representation of coded frames which can be processed to ^ dimensional array 500 stored in block data field 277 (FIG. 
form a moving video image (ie., a moving picture), A 3). The two-dimensional array 500 (FIG. 5) includes eight 
typical sequence of frames is IPPP . . . PIPPP, where I rows and eight columns. Each entry in the array coxFesponds 
indicates an intra-coded frame, and P designates a to a specific entry group, such as Ooup 506, Group 507, or 
predictive-coded frame. For each frame 221, the coded bit Group 508. The groups are based upon the relative iinpar- 
stream representing the frame includes a header 263 and ^ tancc of the entries contained therein. Each group includes 
coded data 265. Each header 263 includes a start code and entries conforming to a specific range of importance levels, 
data related to the respective frame (i.e., picture). In an These importance levels relate to the relative extent to which 
H.261 system environment, much of the header inf<aination the elimination of a particular entry would degrade the 
is required for synchronization purposes. For exan^le, at the quality of the overall video image in a given frame. In the 
frame (picture) layer for frame 221, header 263 includes a example of FIG. 5. Group 506 is the most important group, 
picture start code (PCS) field 267, a picture number (JR) and includes entries having a relatively high level cf impor- 
field 269, a picture type (FTYPE) field 271, a PEI field 273, tance. CSroup 507 includes entries having an intermediate 
and a PSPARE field 274. The PEI field 273 and the PSPARE level of importance, and Group 508 includes entries having 
field 274 are adapted to accoimnodate extra information the least importance to the overall quality of the video 
which may be required for future applications. ^ image. 

Picture data is segmented into Groups or Blocks (GOB) FIG. 6 is a software fiowchart setting forth a procedure fcr 

223, 225, 227, 229, 231, 233, 235, 237, 239, 241, 243, and perfonning video bit rate matching according to a preferred 

245. A GOB (for example, GOB 229) comprises one-twelfdi embodiment disclosed hdrein. The operations set f orih in &e 

of the coded frame (OF) 221. Therefore, GOB 229 may be flowchart may be in^lcmcntcd using the hardware previ- 

conoeptualized as including one-third of one quarter of a 45 ously described in connection with FIG. 1, wherein the 

coded picture area. The arearepresented by one-quarter of a functioning of rate control 113 circuit was disclosed. As 

coded picture is abbreviated as QCIF. Accordingly, diere are shown in FIG. 6, the video bit rate matching procedure 

12 C}OBs 223, 225, 227, 229, 231, 233, 235, 237, 239, 241, consists of six steps. The first stq} is initializatioa 600, 

243, 245 in a CIF frame 221, and three CjOBs in a QCHF followed by macro block piocessmg 601, macro block 

frame. The arrangements of (jOBs in a QF/QCIF picture are 5^ parameter updating 602, and macro block counting 603. At 

depicted in FIGS. 2 and 3. block 603, a test is pezformed to ascertain whether or not the 

Each GOB 229 includes a header field 291, foUowed by number of macro blocks that have akeady been processed 

a macro block data field 298. The header field 291 includes arc greater than a maximum number of macro blocks. If not, 

a GOB start code (GBSC^ field 292, a group number (GN) the program loops back to block 601. If so, the program 

field 293, a group type (CjTYPE) field 294, a GOB quantizer 55 progresses to block 604, where frame parameters are 

(GrQUANT) field 295, and spare infcnnation fields in the updated. Next, at block 60S, frames are counted, and a test 

form of GEI field 296 and GSPARE field 297. Each GOB is performed to ascertain whether or not there are additional 

229 consists of 33 macro blocks, such as **macro block 24" frames to consider. If so, the program loops back to block 

(reference numeral 247) and **macco block 25" (reference 601. If not, tiie program ends. 

numeral 249). The arrangement of macro blocks within a go Referring to block 600, in the initialization stage, the rate 

(jOB is depicted in HG. 2. control unit 113 (FIG. 1) performs the following series of 

Each macro block includes a header field 275 followed by operations: 

a block data field 277. The header field 275 includes a macro 1. Obtaining a value for the desired video ouq)ut bit rate, 

block address (MBA) field 279, a block type inf oimation This value may be received, for example, from one or mc^e 

(MTYPE) field 281, a quantizer type (MQUANT) field 283, 65 endpoint devices; 

a motion vector (MVD) field 285, and a coded block pattern 2. Specifying the maximum ou^ut frame rate based upon 

(CBP) field 287. The block data field 277 of each macro the desired video output bit rate; 
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3. Sending the maximum output frame rate to one or more dse if differenceO, increase the number of indices to be 
enc^int devices; i.e., senda endpoint devices— which arc set to zero. 

to be used to send video infcHmation to other endpoint dsc no change. 

devices. Once the sender endpoint device receives die maxi- At the end of processing each maao blocks the macro 
mum output frame rate, this endpoint device is farced to 5 block counter is checked against the total number of mblk to 
encode video signals with the specified maximum frame ascertain whethw- or not a frame is finished. Jf a frame is 

finished, rate control 113 circuit starts updating the frame 

4. Computing the average number of bits per frame and parameters. At block 604, die rate control 113 circuit per- 
stores this number in a memory location designated as forms the following series of operations: 
averagc_bits_per_fraiiie. If the desired video output bit 1. Obtaining the transmission buffer 111 status; 

rate is R^^ and the maximum frame rate is F^„„ then the 2. Obtaining the total numba" of bits used in the frame; 

3 Based on the targeted memory location occupancy rate 
av««g<^bit£L_4«r_W,=4=;^- for transmission buffer 111 (Le., buffer fulkess), computing 

the targeted bits for the next frame and the taigeted bits fcff 
15 each macro block; 

5. Initializing the transmission buffer with an initial buffer ^ ^" targeted bits fw eadi maao block, 
fullness specified as Bq. providing an appropriate control signal for the 1st maao 

^^^pecifying the targeted_bits_pcr^amc for the 1st SISfrL^VSi^rupdating^ the new frame is checked. 

20 If there are no more bits, then the procedure ends. 
Otherwise, the procedure reverts back to processing macro 

tar^ted_bks_pcr_&Hme = JT* blocfcs. 

A second method of video bit rate matching is the 
«,i,Ar* ir w u u ^ ^ i. . ^quantization of DCT coeflBcients. The outaut signal at the 

m^rfr^Jf. f^^^'^l^'^ ^"^^ 105 of VMD 104 includes t^o coiiponents: 

StS Mf!^^ "^"^ ^^''^^ quanti^DCTcoeffidents,andaquantizationpairetcr.In 

targeted_lnts_pa:_mblkis determine values for the DCT coefficients, an 

inverse quantization operation is performed on the quantized 
targeted-bits^pc^bJk^ ^i't^tT^!^^ ^ coefficients as foUows. Let {x„ i=0,l,2, ... «3} be the 

30 quantized DCT coefficients and {y,, i=0, 1, ... 63} be the 
reconstructed DCT coefficients, witfi Qp representing the 
7. Accordmg to the targeted_bits_per_jnblk, spedfying quantization parameter. Then, with respect to an H.261-like 
the particular group indices which are to be set to zero. environment, in the lading mode, the reconstructed DC 

At &c macro block processing step (HG. 6, block 601), coefficient yo is calculated using (he relationship 
the DCT coefficients processor 107 (FIG. 1) performs the 35 
following steps: 

1. Receiving a macro block from the VMD 104 (FIG.1); yo^*8, 

2. Receiving a control signal over signal line 114 from and the remaining coefficients are calculated using the 
rate control 113 circuit in the control signal specifies one cr formula 

mere DCT coefiScients to be set to zero; and 40 

3. Setting the DCT coefficients specified by the control 

1 to zero. y^[jr,*2+siQ Cx|)]*0f. 



After processing one macro block, the rate control 113 wherc{i=l,2, . . . 63} in I mode and {i=0, 1, . . . 63} inP 
circuit receives the new buffer status from transmission mode, and the sign(w) fiinction is deflned as follows- 

buffer 111 via first input/output 112. The rate control 113 45 

circuit uses the buffer status, defined above, to update the / j if w S 0 

control signal. Since the transmission buffer 111 is used as siff^iw)^ | -qp l^yy^i) 

a teinporary storage facility for video information, the buffer 

^yoperatcinafir^tstate,wherethebufferisi^ control the amount of data produced bv the DCT 

ofreceivxng new video infonnation from input buffer no cr 50 ^*ffi^- ^Z, L^^t^ ^ 

ouQ)ut this infonnation to buffer output 116 HicnZTft^ ^ ^ and sends it to the DCT 

^.r. eu. X. iT^T ^ . . coefflcacnts processor 107 to requantize the DCT coefiB- 

status of the buffer refers to tbc state in which die buffer is „>ntt I m I, i tax uHT ^ 7r!^ 

operating at a given moment 55 Sl-^. ^^'h ' ' kI ^ 

The stq,s implemented by rate control dnnrit 113 for ^^^S^ ^\''Z'^^'^TT ^'^^ 

updating the control signal iidude: ^^^^ ^ cu:cuit Then, the new 

1. Obtaimng the tot^ number of bits used for the mblk. coefScents arc determined by 
bits_j)cr_mblk; 

2. Computing the difference between the targetcd_bits_ 60 v<yo+4y8, 

^l^'^^f^^tn^il^r'^^^^^ wii. u: "^^^'^ ^ cocfidcnt of the I-codcd macTo block 

^ hits_difference4«targeted_bus jer„mblk-bits_per^ ^^^^ coefficients are obtained by 

3. Updating the control signal on signal line 114 based on 

the following: 55 2r^P*CaSp«*) 

If diffcrence>0, reduce the number of indices to be set to where {i=l, ... 63} for the intra-codcd macro block, and 

{i=0,l, . . . , 63} for inter-coded macro blocks. 
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With respect to the second method of matching video completed. If a frame has been completed, rate control 113 

signal bit rates , the sequence of operations perf onned by the drctdt commences updating the frame parameters. At block 

hardwareconfigurationof FIG. lis virtually identical to the 604, the rate control 113 circuit performs the following 
process set forth in FIG. 6, with the following exceptions. 



Referring back to FIG. 6» at the initialization stage (block 5 1. Obtaining the transmission buffer 111 status; 

<rOO), the rate control 113 circuit performs the following 2. Obtaining the total bits used by the fi^e; 

steps: 3. Based upon the targeted buffer fullness (memory loca- 

1. Obtaining the new (desired and/or target) video output tion occupancy rate), computing the targeted bits for the next 
bit rate; frame and the targeted bits for each macro block; 

2. According to the new video output bit rate, specifying 4. Based on the targeted bits for each macro block, 
the maximum output frame rate; generating an appropriate control signal for the first macro 

3. Sending the maximum frame rate via communications block of the next frame. 

line 115 to the sender endpoint device (defined above) to After frame parameter updating^ the new frame is 

force this endpoint to encode the video with the maximum <=hecked. If there are no more bits then the procedure ends, 

frame rate Otherwise, the procedure reverts back to the maao block 

4. Computing the average number of bits per frame as P'^^^^f ^^^^^^^^f , . ^ ^ , . ^, 

7.^ ^ ^ % . , ,^ K.v%.„*-„ u^-o A thurd method of video bit rate matching may be 

avecage.bits jer^e. Let new video bit rate be R,,, ^ conjunction with a preferred embolment di^^ 

and the maxmium frame rate be F,^ then the closed herein. This third method includes all methods which 

^ represent combinations of various features of the first and 

average^bita_peii_&ame«=-T^ . 20 second methods. The manner in which the first and second 

methods arc combined is determined by the specific appli- 
cations of a given system One illustrative combination of 

5. Initializing die transmission buffer with an initial buffer ^^^^^ methods is the process of using DCT 
memory occupancy rate (fuUaess) of Bq* coefiQcient partitioning to process intra-coded macro blocks, 

6. Specifying the targeted_bits jer Jrame for the 1st 25 and then en^loying requantization to process the intcr- 
frame. cocted macro blocks. 

Although system 100 (FIG. 1) with the DCT processor 

targeted_J)itsL.p«_fraine=i:*-^ equipped fOT hnplemcnting the three different processmg 

schemes described above is satisfactory for lower rate 

30 reduction and intra-coded frames, there is a mismatch, 

where K is a constant which is chosen based on the maxi- "driff * between an endpoint device that transmits video 

mum frame rate and the initial buffer fullness Bq. Then the infonnation at a fast rate relative to other endpoint devices 

taigeted_bits_per_jiiblk is which decode this video information at a slower rate. This 

mismatdi is brought about because the video encoder is 

ur£acd_bit.^,ju_jDblk ^ 1*" required to opotate at a fastCT bit rate than the video decoder: 

gcica_uu5»pa- totaL_Qumbcr_^iLjnbik mismatch exists for an the inter-coded frames and is 

likely to accumulate with time^ unless an intra-coded frame 

7. According to the targeted_bits_^per__mblk, specify the is periodically inserted into the video bit stream. To control 
new quantization parameter QPtunv* the accumulation of the mismatch, an improved DCT pxo- 

At the macro block processing step (block 601), the DCT 40 cesser with the mismatch correction elements is shown in 

coefficients processor 107 performs the following steps: FIG. 7. 

1. Obtaining an mblk from the VMD 104; FIG. 7 is a hardware block diagram setting forth an 

2. Performing inverse quantization based on the Qp and illustrative structure for the discrete cosine transfonnation 
recovering the DCT cocflScients; (DCT) processor of FIG. 1. The hardware configuration of 

3. Obtaining the control signal on signal line 114 fromrate 45 FIG. 7 represents an improvement over flie DCT coefficient 
control 113 circuit; processoa* disclosed before in connection with FKJ. 1, as 

4. Using the control signal on signal line 114, requantizing well as other existing state of the art systems, such as the 
the DCT coeflficients. systems described in an ITU-T document entitled, '*Low 

After the processing of one macro block has been Bitrare Coding (LBC) for Videophone'^docun[lent no. LBC- 

completed, the rate control 113 circuit 113 obtains the new so 94-166. This document describes methods for reducing the 

(current) transmission buffer 111 status and updates the bit rate of compressed video infonnation with a minimal 

control signal on signal line 114. The steps implemented by amount of associated iMX>cessing delays. One implementa- 

rate control 113 circuit include: tion described in the ITU document utilizes one motion- 

1. Obtaining the total number of bits used for the macro compensated prediction storage device and two transform 
block, bits_pcr_jnblk; 55 operations: a forward transform operation, and an inverse 

2. Computing the difference between the targeted_bits_ transfonn operation. The main purpose of this implementa- 
per jiblk and the actual bits__per_jiiblk: tion is to correct the "drift", i.e-, the mismatch, between a 

bits_difference+=targeted_bits_4)eT_jnblk-bits_per_ video encoder and a video decoder, 

mblk. According to a preferred embodiment disclosed herein, 

3. Updating the control signal 114 based on the following: 60 the two transform operations described in the preceding 
If differencoO, reduce the size of the quantization param- paragraph are no longer required. Rather, motion conq)en- 

eter; else if difference<0, increase the size of the quandza- sation is performed in the transform domain, as the terms 

tion parameter; else no change. ^*motion con4>ensation" and 'transfer donoain" are generally 

At the end of processing each macro block, a macro block understood by those skilled in the arL With reference to FIG. 

counter which counts the number of macro blocks which 65 7, one feature ofthis embodiment is that&e drift error signal 

have been processed, is checked against the total number of stored in a picture memory of a prediction frame storage 

macro blocks to ascertain whether or not a frame has been device 703 need not be stored with full accuracy. In 
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particular, only a small number of the lowd-frequency ou^ut of the hit rate matching system of FEG. 1. The output 
components of the transfonn coefficients need to be retained of this hit rate matching system may be provided from the 
in the picture memory. Since only a relatively small number transmission buffer 111 (FIG. 1). Note that the output of 
ofcocffidents are now involved in the motion condensation quantizer 702 (FtG. 7) represents the DCT coefiSdents 
I^ocess, and the transfonn operations are no longer needed, 5 processor ou^ut 108. This output 108 is then recoded and 
implementation of the embodiments disclosed hexein is multiplexed with motion vectcrs and quantization infonna- 
simplified considerably over the system described in the tion VME 109 (FIG. 1). The VME 109 may then send the 
above-referenced ITU-T document identified as no. LBC- recoded, multiplexed signal to transmission buffer 111. The 
^"l^- signal is stored in the transmission buffer 111 prior to 

The sinq)lified system disdosed herein is described below lo transmission at the desired output bit rate. The fullness, or 
with reference to FIG. 7. An improved DCT (discrete cosine buffer memory location occupancy ratio, of transmission 
transformation) processor 107 is shown, which indudes an buffer 111 is used to control the quantization levds for 
inverse quantizer 701, a quantizer 702, and a prediction quantizer 702. 

frame storage device 703. The inverse quantizer 701 accepts Next, assume that the output of inverse quantizer 701 does 
DCTcoeffidents and quantization parameters of an input bit is not equal the output of quantizer 702. The ou^ut 707 of 
stream from the first output 105 of VMD 104 (HG. 1). The inverse quantizer 701 will be denoted as "A", and the output 
ou^ut of inverse quantizer 701 which is the reconstructed 108 of quantizer 702 wiO be denoted as **B'*. Thus, an error 
DCT coeffident, is coupled to a first ii^ut of a summer 704, of B-A is added to the picture data. This error, denoted as 
and this ou^ut is also coupled to a first input of a subtractor Ed, is subtracted from the picture data by the system of FIG. 
706. The output of summer 704 is fed to a first input of 20 7. At initial startup, Ed is zero, and the data pass unchanged 
quantizer 702. A second input of quantizer 702 is under through summer 705 to the prediction frame storage device 
control of the signal line 114 which is coupled to rate control 703. Typically, only a small number of low-frequency coef- 
113 circuit (PEG. 1). ficients are fed to subtractor 706, and thus, Ed is only an 

Hie output of quantizer 702 (HG. 7) which is the approximationof the actual drift error due to requantization. 
re-quantized DCT coeffident 108, is fed to a second input of 25 During receding of the next video frame, Ed is approxi- 
subtractor 706. The output of subtractor 706, which is the matcly equal to the drift oxor of flie previous frame, 
difference between ou^ut 707 and output 108, representing During motion-coii^ensated prediction, prediction frame 
die DCT coeffidents of the error signals; Le,, "drift" signal, storage device 703 uses motion vectors on the second output 
is connected to a first input of summer 70S. The output of 106 of VMD 104 (HG. 1) to ou^t a displaced drift error 
summer 705 is coupled to a first input of prediction frame 30 signal. As the prediction is performed direcdy in the DCT 
memory storage device 703, and a second input of predictioD domain, the ou^ut of the prediction frame storage device 
frame memory storage device 703 is connected to the second 703 representing the displaced drift cirar signal, is repre- 
ou^ut of VMD 104 (HG. 1). The output of prediction frame sented by its DCT coefficients. 

storage device 703 is fed to a second input of sununer 704 To compute the displaced drift error direcdy in the DCT 
and this ou^ut is also fed to a second input of summer 705. 35 domain, the following operations are perfarmed by fee 
Inverse quantizer 701, quantizer 702, summers 704, 705, prediction frame storage device 703. Since the motion 
and subtractor 706 are system components whidi are well- vectcr 106 represents an arbitrary number of pds, and the • 
known to those skilled in the art Conventional components DCT representation of the reference frame stored in predic- 
may be used for these items. With respect to the prediction tion frame storage device 703 are grouped into a block- 
frame storage device 703, this device indudes a video buffer 40 based format, the motion conq)cnsated optimal block may 
for storing information conresponding to one or more video overlay with four ndghboring blocks in the DCT block 
frames, a random-access memory device, and a micropro- structure. 

cessor for controlling the operation of the buffer and tfie Let d, D^, D3, D4 be the four neighboring (adjoining) 
random-access memory. The microprocessor is equipped to blocks in the reference frame, and PMC error be the DCT 
execute a software program adapted to perform the steps 45 representation of the displaced drift oror block. Then 
outlined bdow in connection with the prediction frame 
storage device 703. 4 
The hardware configuration of HG. 7 <^crates as follows. ^"^'^£1 
Assume that the quantized DCT coeffidents of an input 

video hit stream having a Mtrate of Rl passes from the first 50 n n ™ tu nm. 

output 105 of VMD 104 (HG. 1) to feTinput of inverse H^nJw ^^^^CTr^f -^tauons of the spatial 

quantizer 701 (FIG. 7), At inverse quantizer 701, the quan- ^"^"^ "'^ 

tified DCT coeffidents are re-constracted to produce the r i r i 

DCT coeffidents. Hie DCT coefficients phis the DCT coef- » = r ^ - K I 

fidents of the "driff* error are sent to DCT coefficient 55 " r" ^ J L ^ ^ J 

processor 107. One purpose of DCT coeffidents processor 

107 (FIGS. 1 and 7) is to generate an output signal r^re- where h and w rroresent overlay width, 
sentmg the processed coeffidents. When the DCT coeffi- The G^ and G,, can be prccomputed and stored in 
acnts processor 107 is imtiaUy started up, there is no drift prediction frame storage device 703. As the drift error signal 
OTor between output 707 and ou^ut 108 (FIG. 7). 60 contams a lot of zeroes, the required cornputation is er«itiY 
-Dierefore, upon initial startup, inverse quantizer 701 pro- reduced. Further, if motion vectors are zero, or Stegor 
vidcs an output signal induding reconstructed DCT multiples of the block width, the above block adjustment 
coeffiaents, and diis signal passes, undianged through sum- procedure can be avoided. This drift error signal will be seen 
' T^f"' ^' . coeffidents processor ou^ut 108, and at trans- 

• 65 mission buffer 111 (FIG. Dwhidirecdvesbitsatthedcsircd 

si^al hne 114 from Ae rate control 113 drcuit (HG. 1), so ou^ut bit rate. Wthout correction, this drift error will 
astoprovideasignalhavmgthedesircdoutputbUratcatthe accumulate over time and eventuaUy result in unacceptable 
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system pcrf oimancc. In order to ameliorate the problem of An exan^Le of a suitable end^wint device is an ITU-T 

drift eirar accumulation, the previous £rame motion c<Kn- H.320 audiovisual tenninal, but any device capable of 

pensated drift enror Ed is added to the present frame signal terminating a digital multimedia stream and presenting it to 

(output 707) prior to requantization by quantizer 702. If the user constitutes an ent^int device. A particular product 

quantizer 702 introduced very little error, this would com- 5 exan^)le of an H.320-compatible endpoint is the AT&T-GIS 

pletely correct the drift error accumulation problem. vistimn. 

However since quantiz^ 702 mtroduces a finite amount of computer-controUed device which includes 

e^or thedriftcanoidybepartiaUy cone^^^ multipUcity of coZnunications ports, such as first com- 

of subtracter 706 will not, m general, be zero. Thus, summer « «»*it*K"vi j v« w^iiu«x 4/ ' . ^. ^o^>» 

705 adds the drift error from the current frame to the ^„ mumcalions port 870 and second com^ 

approximateaccumulateddiifterrorfrompreviousframesto which may be selecuvely interconnected m a vari^ of ^^^^ 

produce an approximate accumulated drift error Ed for the to provide oommunicatoon among a group of endpoint 

current frame. devices 801, 802, 803. Although the system of HG. 8 shows 

The prediction frame storage device 703 only has to two conmuinications ports, this is done for illustrative 

confute a small number (Le., N) of compensated coefiS- purposes, as any convenient number of communications 

cients. Note that, for intra-blocks of video data, the predic- ports may be employed. MCU 810 also includes a control 

tion frame storage device 703 is programmed to set Ed to processor 840, an audio processor 841, a video processor 

zero. The relatively small number of computations required 842, a data processor 843, and a common intenial switch 

to implement the methods disclosed herein is vastly reduced 819. Each communications port includes a network 

as contrasted with the relatively large number of con^Duta- interface, a demultiplexer, and a multiplexer. For example, 

tions required to perform existing processes using pel 20 grst communications port 870 includes network interface 

domain motion compensation. An additional advantage of 813, demultiplexer 813, and multiplexer 822. 

the disclosed m^ods is that these methods require much Although MCU 810 is shown with two communications 

less memory space than existing prior art methods. f^j- purposes of illustration, MCU 810 may. 

The video transmission rate reduction system shown in ^^^^^ convenient number of coimnunications 

FIG. lean be implemented, fOT example, by using a general- 25 ports. For an MCU 810 having N ports, there are N network 

purpose micr^occssor, a digital signal processor (such as interfaces, one control processor, one audio processor, one 

an AT&T DSP 3210 or an AT&T DSP 1610), and/or a video processor, and one data processor. For each processor, 

programmable video processing chip (such as an integrated ^ ^ ^^^^ ^^^^^ from N demultiplexers and 

orcuit known to those skiUed in the art as the UT VCP ^^^^^ ^^^^ ^o the N multiplexers. Therefore, 

^P)* MCU 810 may be conceptualized as an N-port MCU where 

Multimedia System Using Video Processing of the only two communications ports 870, 872 are explicitly 

Present Invention shown. 

To illustrate various typical applications for the present As shown in FIG. 8, first communications port 870 

invention in the context of multimedia conferencing, FEG. 8 35 includes network interface 811, demultiplexer 813, and 

shows a multimedia system using a video processor multiplexer 812. Network interface 811 is a circuit which 

embodying the hit stream rate matching techniques dis- provides the conversion function between the standard line 

closed herein. Referring now to FIG. 8, a block diagram signal codmg used by ISDN network 804 and the Px64 kbps 

setting forth the system architecture of a multimedia con- H.221 signal used by MCU 810. Netwcffk interface 811 

ferencing system 800 is shown. The conferencing system 40 includes output port 812, which is adapted to provide an 

includes an MCU 810, an ISDN network 804, and a plurality output in the form of an H.221 signal The H.221 signal is 

of endpoint devices sudi as first endpoint device 801, second actually a multiplex of sevaal difercnt types of inf onnation 

endpoint device 802, and third endpoint device 803. (audio, video, data, control); therefore, network interface 

Endpoint devices 801, 802, and 803 are coupled to MCU ^^t send the incoming MCU H.221 signal to a demul- 
810 via ISDN networic 804. These endpoint devices 801, 45 ^l^^S ^^ice such as demultiplexer 813. Likewise, net- 
802, and 803 may include one or more user interface work interface 811 has an input port 823 adq)ted to receive 
devices. Each interface device includes either an input ^ ^.221 signal from multq>lexer 822. Mult^lexer 822 
means, an output means, or an input means combmed with combines a pluraUty of individually-processed signals 
an output means. Output means are ad^ted to convert which are to be transmitted to a particular endpoint device, 
multirnedia electronic signals representing audio, video, or so Demultiplex^ 813 separates an incoming multimedia 
data into actual audio, video, cff data. Input means are signal stream received from netwcffk interface 811 into four 
adapted to accept audio, video, and/or data inputs, and to con^nents: a first con^onent 814, comprising electronic 
convert these inputs into electronic signals representing signals representing control; a second component 815, corn- 
audio, video, and/or data. Examples of user interface devices prising electronic signals representing audio; a third corn- 
include video display, keyboards, microphones, speakers, 55 ponent 816, comprising electronic signals representing 
and video cameras, or the like, video; and a fourth component 817, representing data. The 

Endpoint devices 801, 802, and 803 are adapted to first second, third, and fourth conqwncnts 814, 815, 816, 

communicate using existing multimedia communication ^l"^ represent outputs of demultiplexer 813 which are 

protocols such as ISDN. The endpoint device multimedia coupled to common internal switch 819. 

communication protocol controls the presentation of media 60 Multiplexer 822 accepts a plurality of incoming multime- 

streams (electronic signals representing audio, video, and/or dia signal components from common internal switch 819, 

data information) to the endpoint device user. Bndpdnt such as a first coiiq>onent 818 representing control, a second 

devices may function bi-directionally, both sending and component 891 representing audio, a third con^xment 820 

receiving multimedia information, or, dtcmatively, endpoint representing video, and a fourth oonqionent 821 represent- 

devices may function uni-dircctional, receiving but not 65 ing data. The multiplexer 822 integrates the first second, 

sending multimedia information, or sending but not receiv- third, and fourth components 818, 891, 820, 821 onto a 

ing multimedia information. single multimedia signal stream which is coupled to network 
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interface «U, This single multimedia signal stream may be and then le-encodes the linear sum. For each cndpoint 

conceptualized as the ou^ut of multiplexer 822. The net- device, ihls Hnear sum may be subjected to additional 

work interface 811 routes this multimedia signal stream to a processing steps, so as to provide each endpoint device with 

specific endpoint device 801, 802, 803. For second coramu- audio infonnation specific to that endpoint device. These 

nications port 872, the four output components arc first 5 additionalproccssuigstcpsmayiEchide, for example, any of 

con^nent 824, representing control, second con^onent following: the output sum for a given endpoint device 

825, rc5>resenting audio, third component 826, representing f^X exclude that endpoint* s input; the sum may include 

video, and fourth component 827, representing data. The inputs whose present or recent past values exceed a certain 

four input components to multiplexer 834 are first compo- threshold; or tfie sum may be controlled from a spedally- 

nent 828, representing control, second component 829, lo designated endpoint device used by a person termed the 

representing audio, third component 830, representing "chair", thereby providing a feature generaUy known as 

video, and fourth component 831, representing data. chair-controL Therefore, the output of the audio processor 

Common internal switch 819 contains a pluraHty of ^1 is in the form of N processed audio signals, 

electronic switches, buffers, and/or amjrfifiers under ±e ^ ^® ^^^^ switching, Ihe iaput signal to audio 

control of control processcr 840. Common internal switch 15 processor 841 is a single audio signal which is selected from 

819 is co!5)led to audio processor 841 for mixing and a given communications port 870 or 872, based upon control 

switching electronic signals rei^esenting audio; common signals received firom control processor 840. No audio 

internal switch 819 is also coupled to video processor 842 processing is in^lemented in the present example which 

and data process^ 843 f ot mixing and switdung electronic involves only audio switdiing. The audio input is broadcast 

signals representing video and data, respectively. Therefore, 20 to all other audio processor 841 ou^uts, either automatically 

common internal switdi 819 effectively receives four ou^ut ^ manual control 

components from each communications port 870, 872 and ^^^a processor 843 includes hardware and software 

routes these ou^t components to selected ones of respec- means for inqplementing one or both of the functions gen- 

tive processors (control processor 840, audio processor 841, erally known to those skilled in the ait as '*broadcast" or 

video processor 842, and/or data processor 843) within 23 "MLP". For each type of broadcast data, data ixsput is 

MCU 810. Likewise, common internal switch 819 receives accepted from only one enc^int device at any one time, 

the ou^ut con^nents of each processor in MCU 810 and Therefore, tiie input signal to data processor 843 is the data 

routes these outputs to the multiplexer 822 of each commu- output firom one of the communications ports 870, 872. This 

nications port 870. data output is broadcast to the otiicr cndpomt devices as 

Common internal switch 819 receives ouQ)ut control ^ determined by control processor 840, according to the 
signals firom control processor 840 over signal line 851 , and capa^iUties of specific endpoint devices to receive such data, 
IH-ovides input contrd signals to control processor 840 over ^ capaWlily codes stored in memory units 
signal line 850. Common internal switch 819 receives ou^ut ^^^^ ^OM) of respective endpoint devices, 
audio signals from audio processcr 841 over signal line 853, Control processor 840 is responsible for determining the 
and provides input audio signals to audio proccssca- 841 over correct routing, mixing, switching, format and timing of the 
signal line 852, Common internal switch 819 receives output audio, video, data and control signals throughout a multi- 
video signals firom video processor 842 over signal line 855, niedia conference. The control processor 840 retrieves one 
and provides input video signals to video processor 842 over ®f capability codes from each endpoint device. Capa- 
signal line 854. Common internal switch 819 receives output codes, which, are stored in endpoint device RAM 
data signals from data processor 843 over signal line 857, ^ ^d/or ROM, specify the audio, video, data, and/or control 
and provides input data signals to data processor 843 over capabilities for this endpoint device. Control processor 840 
signal line 856. Control processor 840 provides control retrieves the capability codes fiom all N endpoint devices 
signals to the audio processor 841, video processor 842, and participating in a multimedia conference. These capability 
data processor 843 over signal line 844. ^re stored in a memory unit (RAM) of MCU 810 so 

ISDN network 804 is connected to MCU 810 over signal "^^^^^ processor 840 can cotrectty manage the conf ff- 

line 805. Wthin MCU 810, signal line 805 is paraUel- ^ endpoint devices. This storage may occur, for 

connected to first and second communications ports 870 ^^^^ ^ ^ random-access memory (RAM) device asso- 

872. For example, in the case of first communications port ?^ wim control processor 840. In turn, MCU 810 sends 

870, signal line 805 is connected to network interface 8U. Sfn^S?^.'''^^ ^ ^ communications ports 
Network interface 811 is coupled to demultiplexer 813 over ^ ' ^^t^ that each of the endpoint devices 801, 802, 803 

signal line 812, and this network interface 811 is also T ""^^^^^^ conummicate with MCU 810 at a bit rate 

coupled to multiplexer 822 over signal line 823. Signal line 'l^^niined by MCU 810 and appropriate for tiiat specific 

812 is coupled to the input taminal of demultiplexer 813 ^^^^^ ^^cc 801, 802, 803. To properly control the 

and signal line 823 is coupled to the output terminal of ^ video processing unit 842, MCU 810 sends 

multiplexer 822. different capability codes for different video opera- 

Aii/i;r^ ^r»>^..^ o>#i - 1 J ^ J , ^ . tions. Jf the cndpoints are coupled to MCU 810 via com- 

output sl|«Usfi«maSecoiCM«tionf A W^^^ S^^t J . "TT" ""^"^J 

of MCU «10. R>r an N-port MCU 810, this <^al include ^ "P^*""' "^"^ 

the N audio signals fi^ the demult^lexas^nlS ^ TTT f ^^^^^ 
communicationTportWO, 872. « W ratej^^ed in &e capability code based on aU the 
_^ ... .! ., , 65 dififerent transfer rates. The new capabiUty codes, which 

To nnx the audio, audio processor 841 decodes each of the include die specified nminuim frame rates, are sent back to 

audio Inputs, linearly adds the signals obtained by decoding, all the en(^int devices. In this case, the video bitstreams 
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generated by the endpoint devices will have different trans- incaipatated into video processor 842. The inputs to the 

feriates but the same frame rate. video processor 842, are for an N-patMCU, an the N-coded 

Control processor 840 receives Inputs which ace entered video hit stream obtained from N demidtj,lex«B. The out- 

by conference participants into the*^ user interface of an P"*' «f '^^f^'^^t'^^^'^Z^'^^^^. 

A - ^^^..j^^QAi «A'» Hill ™;T,fK«fy^ 5 HTC pTOcessecl to hEVC diffcTeiit tTaiisfex Tatcs. Thc outputs 316 

eadjKjmt devke 801, m SM Iliese inputs the inputs to the N multiplexers. TTie number of required 

of chair-confrol commands and comirands embedded m bit ^^^^^^^^ reduction units and bit stuffing nirits is 

streamsconfoimmgtotheH^21 standard Commands from proportional to Ac difference in the transmission rates 

endpoint devices are routed to the control processor 840 to ^^^^ ^^^^^ endpoint devices. If there are M different 

ensure the correct distribution of bit streams to the audio, ^^^^ transfer rates involved in a conference, the video 

video, and data processors 841, 842, 843, respectively, to lo processor 842 needs to incorpCErate (M-1) bit rate reduction 

ensure that the correct audio decoding algonthm is used at systems and (M-1) bit rate increasing units. In the configu- 

the inputs to an audio mixer within audio processor 841, and ^^^^^ ^ y^^2 for illustrative purposes, 

to ensure that any incoming dau is sent to a data broadcast ^ produced by video processor 842 are coupled to 

unit or MLP processor withm data processor 843. ^^^^ ^ ^ ^^^^ ^^^^^y ^j^^ ^^,^^1 ^.g^ 

The control processor 84<) also directs the switching of the ^^^i^ control input switdier 905, ou^ut switcher 906, 
bit streams from the audio, video, and data processors 841, transmission rate reduction unit 100 and bit stuffing unit 200. 
842, 843, respectively, to each multiplexer 822, 834, and j^py^ switcher 905 provides N-to-1, N-to-2, and N-to-3 
specifies the audio encoding algorithm used in the audio switching capabilities. Similarly, output switcher 906 pro- 
mixer of audio processor 841, and the algorithm used at each y-^^^ 1-to-N, 2-to-N, and 3-to-N switching capabilities. The 
output from the audio mixer. The bit streams are routed to switching is controlled by control signals on signal line 844. 
and from the various processors 841, 842, 843 by the These control signals are generated by the control processor 
common internal switch 819, which is under control of the 54^) Control processor 840 has control capabilities for 
control processor 840. handling different transmission rates among a plurality of 

Weo processor 842 processes the video signals received ^ en^int devices, 

from the common internal switch 819. The processing may jf transmission rates among all the endpoint devices 

take the foim of switching the video, or matching the video are the same, the control signals sent out by control proces- 

bit rate. In video switching, the video processor 842 receives g^o 544 serves to place video processor 

one selected video signal from the switdi 819, and transmits 542 juto a switching mode. In such a case, video processor 

the video signal to some or all other ent^oint devices 342 works as a buffer and switcher, such that the input 

partidpatuig in a given multimedia conference. Video selec- switcher provides an N-to-1 switch and the output provides 

tion may be automatic or under manual control. For i-to-(N-l) switch. A direct signal path between input 

instance, the audio processor 841 and the video processor switdier 905 and output switcher 906 is utilized, whereas 

842 may be automatically controlled by control processM- y^^^ transmission rate reduction unit 100 and bit stuffing 

840, such that an endpoint device with currently active audio ^ ^ 2OO are not used. 

(ie., an endpoint device used by the 'Resent speaker which ^ transmission rates among the endpoint devices are 
provides an audio signal to MCU 810 above a predetomined different, and if there are more than two different tiansmis- 
audio ampUtude threshold) receives tiie picture of the end- ^ ^ endpoints, extra rate reduction units 
point device which previously had active audio (i.e., an |lOO and extra bit stuffing units 200 are needed in the video 
endpoint device used by the '^evious speaker*'), while all ^ processor 842. As an example, FIG. 10 shows a video 
other en<^>oint devices receive the picture of the jKCsent transmission bit rate matching system which uses five end- 
speaker, point devices connected to an ISDN network 804 via a 

A time delay may l>e inccapcffated into the video switch- five-point connection wherein the switching functions of 
ing iniplemented by video processor 842 to avoid execs- video processor 842 are integrated into common internal 
sively frequent video image changes caused by spurious 45 switch 819. The ISDN netw^k 804 is connected to a 
sounds. As in the case of audio switching, video switching five-pert MCU 810, including first comnmnications port 840 
may be controlled directly from a specially-designated end- and additional communications ports 871. A first endpoint 
point device used by a person termed the "chair". If the delay device, endpoint A 901, is connected to the ISDN network 
in the video processor 842 and the delay in the audio 804 via a 384 kbits/s communications link. A second end- 
processor 841 differ by a significant (humanly perceptible) 5^ point device, endpoint B 902, is connected via a 256 Mnts/s 
amount, a compensating delay may be inserted into the link. Third and fourth endpoint devices, end^int C 903 and 
appropriate bit stream to retain Hp synchronization. endpoint D 904, respectively, are each connected to ISDN 

To match videobitrates, video processor 842 incorporates network 804 via 128 kbits/s Imks, and a fifth endpoint 

the video transmission rate matching techniques of the device, enc^int £ SK)5, is connected via a 64 kbits/s link, 

present invention. 55 Therefore, tiie MCU 910 must utilize a video processor 842 

With reference to FIG. 9, the hardware configuration of an having five inputs and five outputs. To produce five outputs 

illustrative stand-alone video processor 842 is shown. This with three different transfer rates, the video processor needs 

video processor 842 may be enc^loyed with the MCU 810 of titiree rate reduction units 920, 921, 922, three bit stuffing 

FIG. 8 or, alternatively, the video processor 842 may exist as units 923, 924, 925, and an optional video buffer. The three 

a stand-alone unit. Such a stand-alone unit may be used to 60 rate reduction units 920, 921, 922 provide respective rate 

match bit rates for all types of digital video inf conation. The reductions of 348-to-256 kbits/sec, 384-to-128 Idjits^sec, 

video processor 842 consists of a video transmission rate and3B4-to-64 kbits/sec. The three bit stuffing units 923, 924, 

reduction unit 100, a bit stufSng unit 200, an input switcher 925 provide respective bit stuffing conv^ons of 64-to-128 

905, and an output switcher 906. If deskcd, the input kbits/sec^ 64-to-256 kbits/sec, and 64-to-384 kbits/sec 

switcher 905 and the output switcher 906 could be combined 65 To provide the proper switching for the inputs and the 

into a single integrated switcher unit, such as common outputs, the control processor 841 has to provide proper 

internal switch 819 (FIG. 8) and, hence, would not be control signals, ff endpoint A 901 is used by the current 



04/22/2004, EAST Version: 1,4.1 



5,687,095 



21 



22 



conference speaker and endpoint B 902 is lised by the 
conference speaker who spoke immediately pricr to the 
speaker using endpoint A 901, then the input switcher 305 
provides an input switching function wherein 5 inputs are 
switched to 4 ouq>uts, such that the coded hit stream from 5 
endpoint A 901 is switched to any one of the rate reduction 
units 920, 921, 922 which provide respective bit rate reduc- 
tions of 384-to-256 kbits/sec, 384-to-128 Jcbits/sec, and 
3S4-to-64 kbits/sec. To send the picture of the previous 
speaker to enclpoint A 901, the coded bit stream from 10 
end^int B 902 is switched to bit stuffing unit 925, which 
pof onus a bit stufiBUig fix)m a rate of 256 kWts/sec to a rate 
of 384 kbits/sec. Via die output switcher 306 (FIG. 8), the 
output of rate reduction unit 920, which perfonns a bit rate 
reduction of 384 kbits/sec to 256 kbits/sec, is routed to 15 
mdpoint B 902, the output of rate reduction Unit 921, which 
performs a rate reduction of 384 kbits/sec to 128 kbits/sec, 
is routed to endpoints C and D, 903, 904, respectively, and 
the output of rate reduction unit 922, performing a reduction 
of 384-to-64 kbits/sec, is routed to endpoint E 905. The 20 
output of bit stuffing unit 925 (64-to-3 84 kblts/sec), is routed 
to endpoint A 901. 
What is claimed: 

1. A con^jressed-domain bit rate reduction apparatus 
including: 25 

(a) a first inverse quantizer having an input and an output; 

(b) a second inverse quantizer having an input and an 
output; 

(c) a prediction frame store; 

(d) a quantizer having an input and an output; and ^ 

(e) frrst, second, and third summing devices; each sum- 
ming device having a first input, a second input, and an 
output; 

the first inverse quantizer adapted to accept an input 
signal including quantized DCT coefficients and quan- 
tization parameters, such that an ouQ)ut signal of recon- 
structed DCT coefficients is generated; 

the output signal being coiq>led to the prediction frame 
store; 

the first invCTse quantizer output signal coupled to the first 

input of the first summing device; 
the prediction frame store generating an output signal 

including a drift error signal; the drift error signal being 

coupled to the second input of the first summing 45 

device; 

the output of the first summing device being coupled to 
the quantizer input; the quantizer ou^t being coi^led 
to the second inverse quantizer input; the second 
inverse quantizer ou^ut being inverted and then 50 
coupled to the first input of the second summing 
device; the first inverse quantizer on^ut being coupled 
to the second input of the second summing device; the 
ou^t of the second summing device being coupled to 
the first input of the third summing device; the second 55 
input of the third summing device being coupled to the 
output signal of the prediction frame store; the output 
of the third summing device being coupled to the 
prediction frame store; the output of the quantizer 
comprising re-quantized DCT coefficients and new 60 
quantization parameters. 
2 A compressed-domain bit rate reduction system 
wherein a first compressed video bit stream having a first 
transfer rate is converted in the compressed domain into a 
second conqtressed video bit stream having a second trans- 65 
fer rate slower than the first transfer rate, the system com- 
prising: 



40 



(a) a video multiplex decoder for demultiplexing and 
decoding the first video bit stream to generate (i) 
quantized DCT coefficient microblocks representing a 
plurality of digitized frequency components for the first 
video bit stream, and (ii) header information specifying 
a starting point for a compressed video image, com< 
pressed video image synchronization information, and 
the location of a given microblock or group of microb- 
locks in the compressed video image; 

(b) a DCT coefficients {Eocessor for receiving quantized 
DCT coefficient microblocks from the video multiplex 
decoder and receiving quantization information from a 
rate control processor to generate a set of processed 
DCT coefficient mictoblocks containing fewer DCT 
coefficients tiian the number of coefficients in the 
quantized DCT cocffident microblocks received from 
the video multiplex decoder, 

(c) a video multiplex encoder for encoding requantized 
DCT coefficient microblocks obtained from the DCT 
coefficients processor, and for multiplexing header 
information with the encoded DCT coefficient microb- 
locks to produce the second compressed video hit 
stream; 

(d) a rate control processor providing control information 
for the DCT coefficients processor, the rate control 
processor comprising a processor for generating new 
quantization information for the DCT coefficients pro- 
cessor according to a transmission buffer status value 
specifying the number of available bits remaining in a 
video transmission buffer, and according to a desired 
bit rate for the second compressed video bit stream, 

wherein each microblock adjoins another microblock at a 
miaoblock boundary, and wherein said DCT coeffi- 
cients processor further includes: 

(a) a first inverse quantizer which, in response to the 
receipt of the quantized DCT coefficient microblocks 
and the original quantization information iiom the 
video multiplex decoder, produces an output in the 
form of reoonstnicted DCT coefficient microblocks 
according to the quantization information; 

(b) a quantizer which accepts the reconstructed DCT 
coefficient microblocks and a DCT feedback signal to 
produce requantized DCT coefficient microblocks 
according to die new quantization infcsmation obtained 
from the rate control processor; 

(c) a second inverse quantizer which receives the requan- 
tized DCT coefficient microblocks and produces recon- 
structed DCT coefficients based upon the new quanti- 
zation information; 

(d) a prediction frame storage device which stwes at least 
one frame, each frame including a plurality of DCT 
coefficient microblocks, and which generates a DCT 
feedback signal based upon the DCT coefficient 
microblocks stored in the frame storage device; 

(e) a first summing device coupled to the output of the first 
inveiBe quantizer, the ou^mt of the frame storage 
device, and the input of the quantizer for adding the 
output of the first inverse quantizer and the DCT 
feedback signal from the output of the frame seepage 
device to generate a sum, and flying the sum so 
generated to the input of the quantizer; and 

(g) a second summing device coupled to the first inverse 
quantizer and the second inverse quantizer for subtract- 
ing die ou^ut of the second inverse quantizer from the 
output of the first inverse quantizer to generate a DCT 
error signal. 
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3. A coir^cssed-domain bit rate reduction system as set 
fojth ID daim 2 further including a third summing device 
coupled to the second summing device and the frame storage 
device for performing compensation on the DCT error 
signal, and wherein the second summing device applies the s 
DCT error signal to the input of the third summing device. 

4. A compressed-domain bit rate reduction system as set 
forth in daim 3 wherein the tihird summing device adds the 
output of the second summing device to the output of the 
frame storage device and stores the sum in the frame storage 10 
device. 

5. A con^sscd-domain bit rate reduction system as set 
forth in claim 2 wherein the output of the second summing 
device is coupled directly to the frame storage device. 

6. A compressed-domain bit rate reduction system as set 15 
forth in claim 3 wherein the output of the second summing 
device is coupled directly to the frame storage device, 
thereby bypassing the third summing device. 

7. A con^ressed-domain bit rate reduction system as set 
forth in daim 2 wherein^ f ot a set of intra-coded microblocks 20 
setting forth DCT coeSidents for a current frame of a 
compressed video image, the DCT error signal from the 
output of the second summing device is stored in the 
prediction frame storage device. 

8. A compressed-domain bit rate reduction system as set 25 
forth in daim 2 wherein, for a set of DCT coeflBcient 
microblocks that are intra-coded, the output of the first 
inverse quantizer is coupled to the input of the quantizer, 
thereby bypassing the first summing device. 

9. A compressed-domain bit rate reduction system as set 30 
forth in daim 2 wherein, for a set of DCT coeffident 
microblodcs that are inter-coded, the output of the first 
inverse quantizer is added to the output of the frame storage 
device and the sum is £^plicd to the input of the quantizer. 

10. A oomprcssed-domain bit rate reduction system as set 35 
forth in claims 3, 4, 5, 6, 7, 8, or 9 wherein said DCT 
coeffidents processor further indudes: 

(a) a first inverse quantizer which, in response to the 
receipt of the quantized DCT coefSdent microblocks 
and the quantization information from the video mul- 
tiplex decoder, produces an output in the form of 
reconstructed DCT coeflSdent microblocks according 
to the quantization infomoation; 

(b) a quantizer which accepts the reconstructed DCT 
coefficient microblocks and a DCT feedback signal to 
produce rcquantized DCT coeffident microblocks 
according to the new quantization information obtained 
from the rate control processor, 

(c) a second inverse quantizer which recdves the requan- ^ 
tized DCT coeffident microblocks and produces recon- 
structed DCT cocffidcnts based upon the new quanti- 
zation information; 

(d) a [H'ediction frame storage device which stores a fr^me 
including a plurality of DCT coeffident microblocks, 55 
and which generates a DCT feedback signal based upon 
the DCT coeffident microblocks stored in the frame 
storage device; 

(e) a first summing device coupled to the ou^ut of the first 
inverse quantizer, the ou^ut of the frame storage 
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device, and the input of the quantizer far adding the 
ou^ut of the first inverse quantizer and the DCT 
feedback signal from the ou^t of the frame storage 
device to generate a sum, and ^plying the sum so 
generated to the input of the quantizer, and 
(f) a second summing device coupled to the first inverse 
quantizer and the second inverse quantizer for subtract- 
ing the output of the second inverse quantizer from the 
ou^ut of the first inverse quantizer to generate a DCT 
error signal, and for applying the DCT error signal to 
the input of a third summing device. 
11. A DCT coefficients processor for use in a compressed- 
domain bit rate reduction system wherein a first set of 
quantized DCT coeffident microblocks is converted into a 
second set of quantized DCT coeffident microblocks in the 
compressed domain, the second set of quantized DCT coef- 
fident microblocks having fewer DCT coeffidents than the 
first set of quantized DCT coefficient microblocks, the DCT 
coefficients processor comprising a processing device for 
receiving the first set of quantized DCT coeffident microb- 
locks and quantization infomiation to generate the second 
set of quantized DCT coefficient microblocks, the DCT 
coeffidents processor further including: 

(a) a first inverse quantizer which, in response to the 
receipt of the quantized DCT coeffident microblocks 
and the quantization information, produces an ou^ut in 
the form of reconstructed DCT coeffident microblocks 
according to the quantization information; 

(b) a quantizer whidi accepts tiie reconstructed DCT 
coeffident microblocks and a DCT feedback signal to 
produce requantized DCT coefficient microblocks in 
response to the receipt of new quantization informa- 
tion; 

(c) a second inverse quantizer whidi receives the requan- 
tized DCT coefficient microblocks and produces recon- 
structed DCT coeffidents based upon the new quanti- 
zation infomiation; 

(d) a prediction frame storage device which stores a frame 
including a plurality of DCT coeffident microblocks, 
and which generates a DCT feedback signal based upon 
the DCT coeffident microblocks stored in the predic- 
tion frame storage device; 

(e) a first summing device coupled to the output of the first 
inverse quantizer, the ou^ut of the frame storage 
device, and the input of the quantizer for adding the 
output of the first inverse quantizer and the DCT 
feedback signal from the output of the frame storage 
device to generate a sum, and applying the sum so 
generated to the input of the quantizer, and 

(f) a second summing device coupled to the first inverse 
quantizer and the second inverse quantizer for subtract- 
ing the output of the second inverse quantizer from the 
output of the first inverse quantizer to generate a DCT 
error signal, and for applying the DCT error signal to 
the input of a third summing device. 
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